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Abstract 


In this report, we characterize experimentally a scanning laser rangefinder that employs active 
sensing to acquire three-dimensional images. We present experimental techniques applicable to a 
wide variety of laser scanners, and document the results of applying them to a device manufactured 
by Perceptron. Nominally, the sensor acquires data over a 60° x 60° field of view in 256x256 
pixel images at 2Hz. It digitizes both range and reflectance pixels to 12 bits, providing a maximum 
range of 40m and a depth resolution of 1cm. 

We present methods and results from experiments to measure geometric parameters including 
the field of view, angular scanning increments, and minimum sensing distance. We characterize 
qualitatively problems caused by implementation flaws, including internal reflections and range drift 
over time, and problems caused by inherent limitations of the rangefinding technology, including 
sensitivity to ambient light and surface material. We characterize statistically the precision and 
accuracy of the range measurements. 

We conclude that the performance of the Perceptron scanner does not compare favorably with 
the nominal performance, that scanner modifications are required, and that further experimentation 
must be conducted. 




Chapter 1 
Introduction 


Many tasks and problems require three-dimensional information about the environment. For 
example, mobile robot navigation systems require this information to avoid obstacles, and to 
achieve high-speed mobility [6] . Similarly, the performance of object recognition systems improves 
dramatically given three-dimensional data [3]. 

A rich variety of passive vision techniques produce three-dimensional information [8]. Tra- 
ditionally, they have lacked robustness and generality, and have not proven themselves effective 
in practice. Passive stereo vision is a particularly promising source of range information, but 
requires substantial data processing to match images with each other and to determine range by 
triangulation. 

Active sensing techniques promise to simplify many tasks and problems in machine vision [7]. 
Active sensors transmit some form of energy into the environment, and sense the reflected signals; 
examples include radar, sonar, structured light, and scanning laser rangefinders. Active sensing 
techniques can provide range data with less computation, and can be relatively insensitive to 
illumination conditions (e.g., they can operate at night). 

Several authors have surveyed the active sensing research conducted by the robotics and vision 
communities. Jain and Jain [7] report on emerging themes and research issues in the analysis 
and interpretation of range images, and present a comprehensive bibliography. Besl [2] examines 
a wide variety of range imaging technologies, and compares them quantitatively by evaluating 
a figure of merit based on range accuracy, depth of field, and image acquisition time. Everett 
[4] surveys collision avoidance and ranging sensors for mobile robots. Nitzan [9] assessed range 
sensors for diverse robotic applications, 

From these surveys and our own observations, we perceive both a great potential for active 
sensing technology, and a relative lack of practical experience with it on the part of robotics 
and machine vision researchers. Consequently, we set out to study one particular active sensor, a 
scanning laser rangefinder manufactured by Perception, which we employ currently in our research 
on mobile robot navigation systems [1, 12]. 

A number of researchers [5, 13] have studied a very similar sensor manufactured by Erim [15]. 
Because these papers concentrate on larger problems in robot navigation, they omit (rightfully) 
many practical details and results concerning the sensor, and do not address at all some important 
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implementation topics. We intend to focus on the practical details (in the spirit of Watts et al. [14]), 
and to address topics of practical consequence that have received little or no attention to date. For 
the purposes of this report, our specific objective is to examine experimentally the Perception scan- 
ner’s performance by conducting experiments to identify its geometric, qualitative, and statistical 
characteristics. As part of this effort, we aim to develop methods to identify sensor parameters and 
techniques to determine whether the sensor is operating correctly. 

We report our methods and findings as follows. Chapter 2 discusses the principle of operation 
and the nominal operating characteristics of the scanner. Chapter 3 outlines the experimental ob- 
jectives and setup, and Chapter 4 defines the geometric parameters related to scanning mechanism. 
The next two chapters address the radiometric parameters related to the electronics and optics of 
the sensor. Chapter 5 examines the qualities we observe about the images, such as image hot spots 
and skewing of the scene. Chapter 6 looks at statistical measures of the image, including drift over 
time, and image stability. Chapter 7 discusses some of the image processing techniques we have 
implemented to compensate for deficiencies of the scanner, and Chapter 8 concludes the report by 
summarizing its findings. 
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Chapter 2 

The Perceptron Scanner 


The Perceptron scanning laser rangefinder is an optical-wavelength radar system 1 , and is compara- 
ble to devices built by Erim [15] and Odetics [10]. The sensor volume is roughly 50x45 x35cc and 
the mass is about 30kg (Figure 2.1). This section outlines the theory of operation of the scanner, 
describes its major features, and defines a number of its geometric and radiometric parameters. 



Figure 2.1: The Perceptron scanner. 


1 Optical radar is also called Lidar, which is an acronym for Light Detection And Ranging. 
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2.1 Principle of Operation 

The Perception scanner operates by sweeping a laser beam across the field of view. A nodding 
mirror changes (tilts) the elevation of the transmitted signal and a hexagonal mirror changes (pans) 
the azimuth. The laser beam is generated by an amplitude modulated laser diode that emits 
180mW of laser power at 810nm (near-infrared). Varying the drive current to the diode modulates 
the amplitude of the laser light. 

A surface in the environment reflects the laser beam back toward the scanner, where receiver 
optics focus it onto an avalanche photodiode. After filtering, an electronic detector measures 
the difference in phase between the transmitted and received signals. The phase difference is 
proportional to the transit time, and therefore the distance (range) traveled by the laser beam. A 
final circuit, implemented as a lookup table, compensates the computed range for variations in 
reflected energy due to different surface materials. 

The scanner forms two images, neither in the optical sense. The image in Figure 2.2 from 
our robot test course consists of several large rocks in the foreground, a sand-covered floor, and 
several boxes scattered in the background. In the range image (the left-hand side of Figure 2.2), the 
pixel values correspond to the range determined by phase differences of the amplitude modulated 
signals. In the reflectance image (the right-hand side of Figure 2.2), the pixel values correspond to 
the magnitude of laser energy reflected. In both, the pixel positions correspond to the laser beam 
direction defined by the mirror angles. 



Figure 2.2: Range and reflectance image pair. 

Phase differences can be determined only modulo 2~. Therefore, an inherent limitation of this 
principle of operation is that it cannot measure range uniquely, i.e., it measures range only to within 
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an ambiguity interval. External constraints (not discussed here) must be employed to resolve that 
ambiguity. 


2.2 Operating Characteristics 

Table 2.1 summarizes the operating characteristics of the sensor as specified by the manufac- 
turer [1 1]. We discuss these in the order given in the table. 


Parameter 

Units 

Value 

Description 

Of ov 

deg 

60 

Vertical FOV 

fy ov 

deg 

60 

Horizontal FOV 

N rows 

pixel 

256 

Number of rows 

Ncois 

pixel 

256 

Number of columns 

A 0 

deg 

0.24 

Vertical angle increment 

A 6 

deg 

0.24 

Horizontal angle increment 

Rambig 

m 

40 

Ambiguity interval 

N range 

bit 

12 

Number of bits for range 

N ref 

bit 

12 

Number of bits for reflectance 

Ar 

cm 

0.98 

Range quantization error 


Table 2.1: Nominal values of sensor parameters. 

To define the parameters related to the scanning geometry, let o and 0 denote the angular 
orientations of the nodding and hexagonal mirrors, respectively. Let 6 S and 0 S be the start angles 
of the two mirrors, and ©/ and Of the final or end angles. Then the vertical field of view is 
Of OV = Of — (p s , and the horizontal field of view is 0f OV = Of — 0 5 . 

Image formation quantizes space by mapping the entire scene to an array of points. Let N rows 
be the number of rows in the image and N co is the number of columns. Assuming that the mirrors 
scan uniformly (equal angles), each angular increment of the nodding mirror is Af, = N , and 

0 rows 

each angular increment of the hexagonal mirror is -A# = N * m _ l . 

Image formation also quantizes depth. Let the number of bits in a range measurement be N range , 
and let R am bi g be the ambiguity interval caused by the phase change 2tt. Then it follows that the 
range quantization error is A R = 

The size of the laser beam is another characteristic of the scanner. We call the intersection of 
the beam with a surface the laser footprint. Suppose that one particular laser beam strikes a surface 
depth discontinuity. Then object surfaces at multiple distances will reflect the laser signal, resulting 
in a phase difference measurement that is not unique and not necessarily meaningful. This corrupts 
the range measurement at the pixel, giving rise to the mixed pixel problem. Mixed pixels occur 
only near object edges. 
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Another factor bearing on the mixed pixel problem is beam divergence, or imperfect collimation, 
due to diffraction effects in the projector telescope. Beam divergence causes the diameter of the 
beam to grow from 17mm as it exits the projector telescope to approximately 50mm at 40m [11]. 
The larger the beam divergence, the larger the laser footprint, and consequently, the more severe is 
the mixed pixel problem. 
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Chapter 3 

Experimental Objectives and Setup 


Our experimental objectives are to characterize the scanner and to identify its geometric and 
radiometric parameters. We answer the following questions about sensor performance: 

Geometry 

• Field of view. 

• Minimum distance. We call the minimum distance the offset distance. This is the actual 
distance corresponding to a range measurement of zero. 

Radiometry 

• Accuracy. What is the difference between computed range and true range? Does the accuracy 
depend on target distance? Are there systematic errors? 

• Precision. Does a static object point project to the same pixel over time? Does the measured 
depth of static object point vary over time? 

• Depth resolution. What is the minimum detectable depth change as a function of target dis- 
tance? We distinguish depth resolution (just noticeable difference) from depth quantization. 

• Angular resolution. What is the minimum detectable width or height of a target? 

Performance 

• Distance. Do range measurements depend significantly on object distance? 

• Ambient light. Do illumination conditions affect range or reflectance measurements? 

• Surface material. For two objects at the same distance from the scanner but with different 
surface materials, do the range measurements differ? 

• Temperature. Do range measurements depend significantly on ambient temperature? 

• Incidence angle. How does beam incidence effect the range measurement? 
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We conduct the experiments inside a 30m - 20m building (highbay). The area has both natural 
illumination that enters through windows, and artificial illumination generated by lamps mounted 
on the ceiling. We will refer to these lamps as spotlights because they are powerful and somewhat 
directional. 

We mount the scanner about 80cm above the floor on a table. We can move a target mounted 
on a stand within the building (Figure 3.1). The stand allows us to adjust the height of the target 
and to rotate the target about a vertical axis. (The rotational degree of freedom will be useful for 
studying the effects of different beam incidence angles.) 

We can mount different targets on the stand. One family of targets consists of planar, cardboard 
slabs of dimension 61cm x53cm. One target is untreated, another is coated with soil, another has 
gravel chips glued to it, a fourth is painted black. The different surface materials exhibit different 
reflectance properties, which range from strong (untreated) to weak (black paint). 



Figure 3.1: Black target on stand. 
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Chapter 4 

Geometric Parameters 


We measure sensor parameters relevant to the geometry of the scanner. Those parameters are the 
origin, the field of view, and the angle increments. 


4.1 Origin 

This section describes a graphical method to identify the geometric origin of the scanner in two 
dimensions, and presents results from one trial. Assuming a flat floor in the experimental area, the 
method is as follows (Figure 4.1). 

1. Adjust the stand so that the target is the same height as the scanner (80cm). As we move the 
target to different positions, it remains at the same height as the scanner, because the floor is 
reasonably flat. 

2. Tilt the scanner until the center of the target projects to row 127 in the reflectance image. 

3. Position the target about 6m from the scanner, so that the target center projects to column 
127 in the reflectance image. 

(a) Move the target 2m away from the scanner (radially) so that the target still projects to 
column 127. 

(b) Mark the floor where the target stands. 

(c) Repeat steps (a) and (b) a total of six times. Thus, the final target position lies about 
16 m from the scanner. 

(d) Draw a line on the floor connecting the marks. Because this line appears as the center 
column of the image, we will call it the vertical line. 

4. Repeat step 3 for columns 0 and 255. 
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+ 30 degree 


Scanner 


Side View 



Figure 4.1: Lines of sight intersect at scanner origin. 

This procedure produces three lines drawn on the floor. Their intersection (point A in Figure 4.1) 
is the projection of the scanner origin onto the floor. As the figure shows, the origin lies outside 
the scanner, several centimeters behind the rear surface of the housing. 

The lines do not intersect exactly at a point. Instead, they form a triangle, whose largest cross 
section is approximately 5cm. One possible cause of this deviation from the ideal is that the center 
of the target does not project to row 127 in all cases, as required in step 2. Instead, we find that 
the image coordinates of the target shift, typically one or two pixels, as we move the target. The 
reason for this may be that the floor is not perfectly flat. 

Although two lines suffice to define the origin, we draw three because more data increases the 
robustness of the procedure. Further, choosing in step 4 the extreme column coordinates helps 
delimit the horizontal field of view (see Section 4.2). 

Having identified and marked the origin, we secure the end of a tape measure to point A, and 
extend the tape into the scene along the vertical line. This defines a “ground truth,” allowing us to 
compare computed range values with distance measurements read from the tape measure. 


4.2 Horizontal Field of View 

To measure the horizontal field of view, we place two targets so that they are visible in the first and 
last column in the image (/ and j in Figure 4.2). As in the previous section, the targets are at the 
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z 



line 


Figure 4.2: Identification of the horizontal field of view. 


same height as the scanner and each other. The (fictional) line joining the two targets, and the laser 
beams passing through them form a triangle parallel to the floor. From this triangle and the law of 
cosines, we compute the angle between the two targets by 


8ij = arccos 




(4.1) 


where m,- and nij are the distances from the origin to the two targets, and dij is the distance 
between them (we measure all distances manually on the ground plane). Because the targets lie 
at the extremes of the horizontal field of view, 0f OV = fy. Assuming that the horizontal angular 
increments are equal, we can compute the horizontal angle increment from 6f OV by 



8fov 

N cols 1 


(4.2) 


The experiment resulted in a horizontal field of view 8 fov = 60.3° and _A o - 0.236°. This 
compares with a stated horizontal field of view of 60° and = 0.235°. 


4.3 Vertical Field of View 

Several different experiments measure the vertical field of view and angle increment. Table 4.1 
summarizes the results, which we discuss below. 
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Method 

Vertical Angle Increment 

Vertical FOV 

Tilting Scanner 

0.1667 deg. 

42.50 deg. 

Using a Plumb Line 

0.1566 deg. 

39.93 deg. 

Using the Programmable Tilt 

0.2083 deg. 

53.12 deg. 


Table 4.1: Vertical field of view results. 

A notable result is that the vertical field of view is not near the stated value of 60°. In discussions 
with the manufacturer, it was discovered that an incorrect constant in the scanner control software 
caused this. Perceptron adjusted their software to correct this. 

Tilting the Scanner 

We place the target on the vertical line and take an image. We physically tilt the scanner about the 
scanner mount axis and take another image. Between images, the target appears to shift along the 
center column. We measure the difference Ar in row coordinates of the same target point. We 
measure with a protractor the change Atilt in the tilt angle. The vertical angle increment and the 
field of view are given by 

t _ Atilt 
A# = ~ . 

_lr 

Of ov — A a (N rows 1) . 

The result value of Of 0v = 42.5°. 

Using a Plumb Line 

We place three equidistant marks along a plumb line, and suspend it from the target stand. Let h be 
the distance between the marks. We position the stand so that all the marks appear along column 
127 of the reflectance image, and the center mark appears at row 127. 

We measure manually the distance d from the origin to the target. The angle subtended by two 
marks is Atilt = tan From a reflectance image, we compute the difference Ar between the row 
coordinates of the projections of the marks. 

With Atilt and Ar we compute the vertical angle increment by Equation (4.4) and the vertical 
field of view by Equation (4.4). The result value of <pf OV = 39.93°. 

Programming the Tilt Angle 

We draw a line on the floor that projects to one row of the image. We change the vertical field of 
view in increments of -A = 5° by programming the scanner. As we tilt the field of view, we count 
the number of rows S that the projection of the line shifts. 


(4.3) 

(4.4) 
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We find that the vertical angle increment is 

_ _ 5° _ 0.2083° 

S 24 rows row 

The result value of <bf OV = 53.12°. The error resulting from the programmable tilt mechanism 
may be due to the imprecise starting position of the nodding mirror and incorrect mirror control 
constants supplied by the manufacturer. In other words, even though we instructed the scanner to 
change its field of view, we are not sure whether the scan will start at the same angular position. 
Therefore, the actual angle increment may not correspond to the field of view change as instructed 
by the programmable tilt mechanism. 

Remarks 

In retrospect, physically tilting the scanner yields the worst measurements. This is due to difficulty 
in accurately measuring the scanner tilt angle. Another problem is that the scanner is rotated about 
its mount axis, which is not necessarily the axis of the scanner. 
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Chapter 5 

Qualitative Characterization 


Many sensor characteristics can be noted by observing the images both spatially and temporally. 
This section discusses several aspects of the sensor images that caused problems. 

5.1 Internal Reflections 

We observe two holes or hot spots in the range and reflectance images. These appear as two white 
blobs in the upper center of the images in Figure 5.1. These are believed to be due to interreflections 
within the Perceptron scanner. 



Figure 5.1: Two hot spots in image. 
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Figure 5.2 shows the effect of a scanner internal reflection on the image. The highlighted area 
in the range image consists of a flat floor. However, magnifying the region (shown in the right-hand 
side of Figure 5.2) shows that the range values are not constant. We have observed variations of 
40-50 range counts within this region. This "smudge" is always located at the same position in 
the image (regardless of sensor orientation). Removal of the sensor exit window eliminates the 
smudge, so we conclude that some internal reflections (reflections of the laser beam off the exit 
window) lead to this problem. 



Figure 5.2: Image smudge due to internal reflection. 
Range image on left, magnified region on right. 


5.2 Shadows 

We observe shadows to the right of high-reflectance objects (Figure 5.3). In the figure, the area 
to the immediate right of the large, white cardboard target has a shadow in the reflectance image. 
As the laser beam scans from left to right across the scene, we believe that this effect is due to the 
avalanche photodiode not being completely discharged, and hence having a "memory" of previous 
pixels along the scan line. 
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Figure 5.3: Shadows to right of high reflectance objects. 


5.3 Skewed Objects 

Although the target was a perfect rectangle on a vertical wall, we observed a skewed target in the top 
of the image (e.g., see the skewed rectangle in the intensity image of Figure 5.4). The wire-frame 
model (black rectangle in the intensity image) shows the correct projection of the rectangle. This 
may be due to the fact that the nodding mirror is still accelerating when it starts scanning. Once 
the mirror reaches a terminal velocity, we do not observe any more skew. 



Figure 5.4: Skewed rectangle in the top portion of the image. 


5.4 Vignette Effect 

The aperture of the scanner causes problems at the bottom of images. The lower comers of 
Figure 5.5 have a vignette or region where the pixel values are incorrect. 
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Figure 5.5: Scanner aperture effect in lower comers of image. 
Range image on left, magnified region on right. 
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5.5 Frame Rate 


The stated frame rate of the scanner is 2 Hz, or a range/reflectance image pair every 0.5 seconds. 
However, when taking images at this rate, we observed that not all images of a static scene appeared 
identical. A series of say 20 images would be the same, the next image would be noticeably lighter 
(i.e. have higher range values), then the next series would be the same as the initial sequence. This 
temporal problem diminished as the frame rate decreased. An experiment that tabulated the range 
measurement at a specific target point while the frame rate increased is summarized in Figure 5.6. 
A trial consisted of taking a range measurement of a fixed target, waiting n seconds (n an integer), 
then repeating 100 times to obtain a mean range measurement. The experiment was then repeated 
for a different delay n. The range measurement does not stabilize until frame rates are less than 
0.5 Hz. It is not clear what causes this phenomenon. 



Figure 5.6: Effect of Frame Rate on. Range Measurement. 


5.6 Mixed Pixels 

Mixed pixel refers to a pixel which lies on the edge of an object and therefore has a range value 
that is a combination of the range value of the background and the range value of the object (due to 
the finite size of the laser footprint). An experiment to examine this effect is shown in Figure 5.7. 
Here the foreground target is a small piece of cardboard, with a larger cardboard background. 

Because it is difficult to see the mixed pixel effect in the image, we plot measured range 
values along columns of the top horizontal boundary of the foreground target (see Figure 5.8). 
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Figure 5.7: Mixed pixel effect setup. 


Points represented by a plus lie on the background and points represented by a diamond are on the 
foreground. At the boundary between the two targets, the range value (represented by a circle), is 
some combination of the foreground range and background range. In essence, there is no sharp 
change in the image corresponding to a sharp discontinuity in the scene. Thus, care must be taken 
when using range values at surface boundaries. In particular, edge detection methods that rely on 
large range discontinuities must be avoided. 

5.7 Range Drift 

To determine if the range values drift over time, we placed the cardboard target at a fixed distance 
from the origin of the scanner, and took many images over several hours without moving the target. 
We observed that range values of a particular pixel drift over time (see Figure 5.9). We placed the 
cardboard target at a fixed distance 8m from the origin of the scanner and took 10. 000 images over 
a 3 hour period. Figure 5.9 shows the measured range value at one pixel plotted as a function of 
time (or image number). 

Since we observed not a random but a linear drift in range measurements, we hypothesized that 
the drift might be due to a change in ambient temperature. To confirm this idea, we ran the same 
experiment under different ambient temperatures. We placed an electric heater behind the scanner 
to control the ambient temperature. 

Figure 5.10 shows range values at a particular pixel point over 14.000 images. Without the 
heater, we observed approximately constant range measurements between images 0 through 8,100 
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Measured Range (cm) 



Figure 5.8: Mixed pixels along horizontal boundary. 
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(the ambient temperature was 70°F). Then, the heater was turned on, and we waited 30 minutes 
until the ambient temperature behind the scanner became 1 13°F. We observed an abrupt change 
in range measurements as shown at time A in the figure. While we were continuously taking 
images, we turned off the heater at time B and observed a gradual increase in range measurements. 
Similarly, we again turned on the heater when the temperature became 70°F (at time C). As the 
temperature increased, the range measurements decreased as shown between C and D. At time D, 
we observed a gradual increase in range measurements as soon as we turned off the heater. This 
could explain the cyclic nature of the data in Figure 5.9 caused by the building cooling system with 
a 1 hour period. 

We conclude there is both a short term drift in range measurements (of about ±15cm, seen in 
images 0 — 8000 in Figure 5.10), and a long term drift associated with ambient temperature. Thus, 
a range measurement is very sensitive to the ambient temperature. 



0 2000 4000 6000 8000 10000 12000 14000 

Image Number 


Figure 5.10: Range values drift due to ambient temperature. 
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Chapter 6 

Statistical Characterization 


This section discusses statistical characteristics of the Perception scanner, including angular reso- 
lution, precision, and accuracy. 

6.1 Angular Resolution 

The angular resolution of a scanner is a measure of the minimum width an object must be to 
be detected by the scanner. Instead of directly measuring the minimum detectable width of a 
target, we use an indirect method based on the fact that the minimum detectable width of the target 
corresponds to the distance (as measured by the number of columns and denoted by A co i umn between 
the target boundary and the portion of the target where the range measurements are approximately 
constant. Therefore, we can determine the minimum width by observing a sudden change in range 
measurements along the target boundary. 

If the depth change is noticed only at the boundary pixel, the angular resolution and the angular 
increment are identical. In practice, however, the receiver optics are designed to have a field of 
view several times larger than the beam size (cf. Section 2.2) 

The experimental setup consists of two targets, one small and one large (approximately 
lmx2m). We place the small target 7m from the scanner, and the large target 2m further (Fig- 
ure 6.1). The small target appears in the image between columns 121 and 135. 

Figure 6.2 shows the mean, over 100 images, of range and reflectance measurements taken 
along one row. Between columns 123 and 133, we observe a constant depth value of 711 cm. At 
the object boundaries, we see a sharp change in the mean extending over 2 pixels (from 120 to 122 
and from 135 to 137). Therefore, at this particular distance, A co i umn is 2. The angular resolution 
A ang ie is given by 

A an gl e — 2A co i umn A(j — 0.941 , (6.1) 

where Ao is the horizontal angle increment. 

We can also estimate the minimum detectable width of an object at a given distance using the 
resulting A ang i e . 
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Figure 6.1: Setup to determine angular resolution. 



Figure 6.2: Range and reflectance statistics across a row in the image. 


23 








6.2 Precision 


Precision experiments determine how much repeated measurements vary. We investigate the 
precision of two quantities: pixel position and range measurement. 

6.2.1 Precision of Pixel Position 

To measure the repeatability in pixel position, we take a series of images of the same scene without 
altering the scanner configuration. We expect a point in the scene to project to the same pixel 
position over those images. 

We position the scanner in front of a vertical wall on which we have drawn white circles (radius 
12cm) surrounded by black squares. We consider regions, because we find it difficult to position 
experimentally a target that maps to only one pixel in the image. We acquire a sequence of images. 
From each reflectance image we extract the white circle by thresholding, and then compute the 
centroid of the circular region. 

Figure 6.3 plots the standard deviation of the circle centroid positions for several row and 
column positions. The standard deviation is the order of 0.025° - for comparison, the column angle 
increment of 0.24° is also plotted. The figure shows that the standard deviation of the centroid 
varies little over time. It also shows that the standard deviation is an order of magnitude smaller 
than the nominal horizontal angle increment, and varies little over different pixels in the image. 

From this we conclude that, in general, the precision of pixel position is high, and that image 
stability is independent of pixel position. 
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6.2.2 Precision of Range Measurements 

In this section, we describe experiments to measure the range precision. We vary the ambient light, 
surface material of the target, distance from the scanner to the target, beam incidence angle at the 
target, and ambient temperature. Under each of these conditions, we take 100 images at each target 
position, and compute the mean and variance of the depth measurements. 


Effect of Ambient Light 

To study the effect of ambient light on sensed range, we place the black target at one distance, 
take 100 images, and compute the mean and variance in range at particular pixels. We repeat this 
procedure for six target distances between 6 and 16m under different indoor lighting conditions: 
during a sunny day, and during a cloudy day both with and without spotlights. 

Figure 6.4 shows that the precision decreases with intensity of illumination. We conclude that 
the brighter is the ambient light, the more the range measurements vary temporally. 




Figure 6.4: Range precision under different lighting conditions. 


Effect of Surface Material 

Surface material has a definite effect on the range measurement. The highlighted region of the 
range image in Figure 6.5 consists of a flat, cement floor (gray color), with a dark grease spot. 
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Even though the distance should be constant, we observe different range values for the grease and 
the floor (shown in the magnified region to the right). We conclude that range measurements do 
depend on surface material, and we now quantify this. 



Figure 6.5: Range measurements differ across surface boundaries. 
Range image on left, magnified region on right. 


To investigate the effect of object surface material, we obtained range measurements from 
a variety of targets. Figure 6.6 shows the mean and standard deviation of range measurements 
for various targets at seven different distances. The left-hand side of the figure shows that the 
mean varies with different surface material. The right-hand side of the figure shows that when the 
returned reflectivity values from the target are large (the cardboard, gravel, and soil targets), the 
range measurements do not vary much, but when the reflected signal from the target is weak (the 
black target) we observe high variance. 

This is a difference between the mean range values in Figure 6.6 and Figure 6.4 for the black 
target. We believe this is due to a change in the scanner’s auto-zero circuitry. See appendix A for 
a discussion. 

Effect of Beam Incidence Angle 

As the distance and incidence angle increase, the area of the footprint increases. As the area of the 
footprint increases, the precision of the range measurement significantly degrades. If we assume 
that the footprint is sufficiently small and a is almost constant inside the footprint, the size of 
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Figure 6.6: Range measurement statistics for different surface materials. 


the footprint is roughly inversely proportional to cos a. A first-order approximation of the range 
precision u range is given by [5] as 

rIj 

Grange ■ ( 6 - 2 ) 

COS a 

To verify this model, we conduct experiments in which we rotate the target to produce different 
incidence angles. To minimize the effect of ambient light on the range precision (see Section 6.2.2), 
we run these experiments at night. 

Figure 6.7 shows the mean of the range measurements as a function of incidence angle and 
distance. We observe a variation in the mean of range measurements of about 10cm. 

For incidence angle 0°, we fit a curve with Grange = K\ R\ e + K 2 . For non-zero incidence 
angles, we fit the curves to Equation 6.2 by using die computed parameters K\ and K 2 and the 
measured incidence angle a . 

Figure 6.8 shows the standard deviation of range measurements with different incidence angles. 
Circles indicate the measured a from the experiment, and the curve is drawn from Equation 6.2. As 
predicted, we observe that the range precision degrades as the beam incidence angle becomes larger 
and as the distance increases. We also conclude that Equation 6.2 is a reasonable approximation 
for the Perceptron scanner for 0° < a < 45° and distance < 14m. 


27 




Figure 6.7: The mean of range measurements for different beam incidence angles. 


Effect of Temperature 

Section 5.7 discussed qualitatively the problem of range drift. We believe long term drift is caused 
by changes in temperature, while short term flucuations are due to noise in the sensor. Figure 6.9 
shows the results of range measurements to a pair of targets over our normal ambient operating 
temperature of 65°F to 86°F. From 65°F to 75°F the mean values and standard deviation of the 
range measurement are roughly constant. Beyond this temperature range, the mean value of 
the range measurements increases with temperature. Surprisingly, the standard deviation of the 
range measurement peaks at about 81°F. We postulate that this is an effect of ambient sunlight. 
These measurements were taken during sunny, summer afternoons, from 12:00 to 6:00PM. The 
temperature of 81°F was reached about 3:00PM (when the ambient sunlight was at a maximum). 
The temperature inside the building increase for several hours, but the ambient sunlight diminished. 

Our experience indicates that scanner remains calibrated between 65 °F - 75 °F. As the temper- 
ature increases, it is necessary to re-calibrate the scanner. In particular, temperatures above 80°F 
render the scanner unusable unless frequent re-calibration is performed. 

Again we note some differences between temperature experiments possibly caused by a change 
in the scanner’s auto-zero circuitry. See Appendix A for a discussion. 

Remarks 

Possible causes of temporal variation of range measurements include the laser footprint and surface 
material. The laser footprint is the intersection of the laser beam, whose shape is a cone, with the 
target surface. Therefore, the size of the footprint increases with distance and with the angle a 
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Figure 6.8: Standard deviation for different beam incidence angles. 








Figure 6.9: Range measurement statistics as temperature increases. 


between the surface normal and the beam (the beam incidence angle). Since the measured range is 
a function of the range values over the footprint, the range precision depends on the distance and 
the beam incidence angle. The range precision may also depend on the amount of light reflected 
from the target, which again depends on the surface material and incidence angle. 


6.3 Accuracy 

To determine the accuracy of the range measurements is to identify the distance between them and 
the “true” ranges. For a target point lying in the direction (©. 8), let R,,,.e be the range measurement 
reported by the scanner and let D«,, $ be the true distance from the geometric origin as read from 
the tape measure (cf. Section 4.1). The sensor can only range objects that are at least 2m from the 
scanner, so Ds„,e > 2. Under ideal conditions, we expect to observe a linear relationship: 

R<p,e = aDos + Po • (6.3) 

Here po is the distance from the origin to the (conceptual) surface corresponding to a range 
measurement of zero — we will call it the offset distance — and a is the slope. We expect that po > 0 
because of the constraint on D 0 ,o and because the geometric origin is behind the scanner. We expect 
that a « 1 , otherwise we do not know the ambiguity interval accurately. 

We define D,, K o as the estimate of the distance from the the scanner range measurement R^.e. 
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This is given by: 


(6.4) 


R,:,.e — Po 

U<u,Q - 

a 

To determine the parameters a and po, we acquire range measurements of targets at known 
distances, and fit a line to the data. The slope and intercept are the parameters we seek. The 
detailed experimental procedure is as follows. 

1. Adjust the stand so that the target is the same height as the scanner (80cm). 

2. Tilt the scanner until the center of the target projects to row 127 in the reflectance image. 

3. Position the target about 4m from the scanner, so that the target center projects to column 
127 in the reflectance image, and the target is perpendicular to the vertical line. (Section 4.1 
defines the vertical line.) 

4. Move the target 2m away from the scanner, along the vertical line, so that the target still 
projects to column 127 and still is perpendicular to the vertical line. 

5. Sense R 0 j 100 times. Compute the mean and variance of the distribution of measurements. 

6. Read D 0 .o from the tape measure (cf. Section 4.1). 

7. Repeat steps (4)-(6) a total of seven times. Thus, the final target position lies about 16m 
from the scanner. 

We follow this procedure using different targets (one untreated cardboard slab, one cardboard 
slab painted black, one cover with gravel chips, and one covered with soil). 

Figure 6.6 plots the results. It shows that the relationship between computed range and true 
range varies with target surface materials. 

Table 6.1 shows the parameters extracted by fitting lines to data points in this graph. We observe 
the slope, intercept, and RMS error to vary with surface material. The RMS error gives some idea 
of the average accuracy of the scanner. Figure 6.10 plots the error D 0 ,e — D 0 .e as a function of 
true distance for various surface materials. It is difficult to discern any trends in the data. This is 
because the sensor noise (a & 10 — 15cm) dominates. 


Material 

a 

Po (standoff, m) 

RMS Error (m) 

Cardboard 

\um 

1.13 

0.08 

msm 

0.983 

1.85 

0.11 

Soil 

1.01 

1.56 

0.05 

Black 

-0.17 

5.9 

0.56 


Table 6.1: A line fit for computing the standoff distance. 
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Figure 6.10: Sensor accuracy for different materials. 
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Chapter 7 

Image Processing 


This report has identified several problems with the Perceptron scanner. By understanding the 
types and sources of deficiencies in the scanner images, it is possible to minimize their effects. 
This section discusses the image processing techniques used to overcome these problems. The 
basic idea is to identify and mark pixels in the range image as being either valid (acceptable pixels), 
or conversely invalid (corrupted pixels). 


7.1 Internal Reflections and Vignette Correction 

Because these effects occur in constant, known positions, a simple range threshold operation can 
tag these pixels. The rectangles in the range image in the left-hand side of Figure 7.1 indicate the 
regions to which the thresholding operation is applied. The right-hand side of the figure shows the 
result of the threshold operation - white implies a valid pixel, black implies an invalid pixel. 

Most of the vignette pixels and the smudge due to the internal reflection (lower right-hand 
comer) have been marked as invalid. 

7.2 Surface Material Correction 

Surface material problems occur when dark or black object appear in the scene. The manifestation 
of the problem is that dark objects may have range values 1 — 2m less than a correct value. We 
process the image to identify all unconnected regions in the image. We define connected not only 
in the sense of adjacent pixels in the image, but also connected in range (two pixels i.j and i + 1 ,j 
are connected if | Rangei j — Range i + \ . y j < e). In the natural, outdoor cases we consider, all objects 
are smoothly connected to each other (i.e. there are no floating objects). Figure 7.2 shows the 
result of a connected region analysis. The range image on the left is processed to detect connected 
regions. The binary image on the right uses black values to indicate disconnected range pixels and 
white to indicate connected range pixels. Most of the pixels of the grease spot (see section 6.2.2 
and Figure 6.5) have been marked as invalid. Also, other grease spot areas and a dark garage door 
at the top of the image have been tagged as invalid. 
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Figure 7.1: Range image and result of threshold operation. 



Figure 7.2: Range image and connected regions. 
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Combining the results of the thresholding and connected region analysis results in the image 
in Figure 7.3. In this image, 63% of the range image pixels have been identified as valid pixels. In 
our experience, this is a worst case - generally a much higher percentage of pixels are accepted. 



Figure 7.3: Range image and valid pixel mask. 


7.3 Minimizing Statistical Variations 

Statistical variations can be minimized by applying filters in both the spatial and temporal domains. 
Spatially, do adjacent pixels have similar range values, or is there significant noise? In the 
temporal domain, we are concerned with a sequence of images having a large standard deviation 
due to thermal noise and range drift. Of major concern are outliers - those pixels in the image that 
have significantly different values from their neighbors. A median filter is used to eliminate these 
outlying points. Table 7.1 summarizes statistics gathered for several planar targets in the image. In 
the spatial domain, we look at the statistics over a 3x3 1 pixel region - in the spatial domain, we take 
a sequence of seven 2 images and and compute the standard deviation of the range measurement. 

From this we note that spatially, there is little noise (on the order of 1-2 pixel counts). Tempo- 
rally, the problem is more severe. We conclude that spatial median filter will do little to improve 
the image quality, but that temporal filter can increase the reliability of the range measurements. 


’This size was choosen to reflea a typical 3x3 median window size - anything larger tends to blur the image. 
2 This length yields acceptable run-time 
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Target 

Spatial a (cm) 

Temporal a (cm) 

1 

1.8 

7.9 

2 

1.8 

6.8 

3 

2.2 

8.0 


1.9 

5.4 

5 

1.3 

5.4 


4.1 

7.1 

7 

1.4 

7.9 


Table 7.1: Comparison of spatial versus temporal variations. 
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Chapter 8 


Discussion 


In this report, we have experimentally characterized the Perception laser rangefinder. We doc- 
umented our experimental methods for evaluating and characterizing the scanner, and recorded 
many experimental results. 

The experimental methods for determining the geometric parameters of the scanner — offset dis- 
tance, field of view, and angular increments — are relatively straightforward and effective. Because 
the methods are simple, and because of further experience using the scanner with the computed 
geometric parameters to build quantitative maps, our confidence in the results is high. We believe 
that similar methods could be applied to similar sensors with virtually no modification, and equally 
good results. In the future, we hope to apply them to the Erim scanner. 

The qualitative characterization of the sensor yielded results that are both surprising and 
disappointing. The existence of blatant flaws, such as internal reflections, vignette effects, skew 
in the upper portion of the field of view, range drift over time, and sensitivity to frame rate, raise 
disturbing questions about the implementation of the sensor. The existence of other problems, 
including the “shadows” appearing to the right of discontinuities, the sensitivity to ambient light 
and surface material, and the mixed pixel effect, arise due to inherent limitations of the rangefinding 
technology. In the future, we would like to extend the qualitative characterization of the sensor 
by more thoroughly decoupling and controlling the different causes and effects. As one example, 
we hope to study the effect of ambient light conditions on range measurements given targets at 
different distances and orientations, and with different surface materials. (For this, we might 
use the controlled imaging environment and high-precision equipment of the Calibrated Imaging 
Laboratory at CMU.) 

The statistical characterization of the sensor was revealing, but perhaps not definitive. From 
the experimental results on precision, we conclude that pixel positioning is fairly precise, and 
that range measurement precision depends significantly on the intensity of ambient light, type 
of target surface material, target orientation with respect to the sensor, and temperature. From 
the experimental results on accuracy, we cannot draw unambiguous conclusions because of the 
difficulties in controlling variables such as temperature and incidence angle. 

Qualitatively comparing the Perceptron scanner to the Erim scanner, the Perception furnishes 
more bits/pixel, larger images, and a higher data rate. We observe that the Perceptron senses 
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more meaningful range values from low-reflectance materials (i.e., it has a larger signal-to-noise 
ratio) and supplies more stable images with less jitter and distortion (i.e., it has a better scanning 
mechanism). In the future, we hope to compare quantitatively the two sensors in a comprehensive 
and systematic suite of experiments. 

Interpreting the entire corpus of experimental results, we find that the performance of the 
Perceptron scanner does not compare favorably with the advertised performance, and that its 
nominal operating characteristics are not always observed in our experiments. We conclude that 
scanner repairs and modifications are required, and that further experimentation must be conducted. 
We are optimistic that these efforts will succeed, making the Perceptron scanner live up to its 
tremendous potential for three-dimensional active sensing. 
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Appendix A 

Auto-zero Circuitry 


Auto-zero circuitry within the scanner uses an internal target (at a known, fixed distance) to 
continually re-calibrate sensor readings. The auto-zero system compares the range measurement 
of the target to its known distance, and adds this error signal to the range image to correct for 
temporal drifting of the range values. During the course of the experiments, it was believed that 
this circuitry was functioning incorrectly, and the scanner was returned to the manufacturer for 
adjustment. This change had a deleterious effect on the scanner’s sensitivity to surface materials. 
Compare the results for a black target from Figure 6.4 and Figure 6.6. The first data set was taken 
before the auto-zero change, the second set of data was taken after. The change increased the 
problem of incorrect range measurements of targets with low reflectivity. 

To hightlight this before and after difference, we show the results of an early experiment (before 
the auto-zero change) to test the effect of surface materials. 

An experiment placed the black target on a larger cardboard target as shown in Figure A.l. 
To minimize any time drift effect, we capture 100 images and compute the mean and standard 
deviation of range measurements along a scan line with two different surface materials. We should 
not observe any depth difference between the two targets along the scan line. 

The left figure in Figure A.2 shows range values along a row in the range image. The untreated 
cardboard target is visible between columns 115-120 and 136-140, and the black target is visible 
between columns 121-135. The range values for the cardboard and for the black target are 
essentially identical. This contrasts with the results from Figure 6.6 (takem after the auto-zero 
change) where the range measurements between the cardboard target and the black target vary 
significantly. After the change, the sensor range measurements became more sensitive to material 
properties. The right figure in Figure A.2 shows the standard deviation of range and reflectance 
measurements along the same row in the image. The range variations are larger for the black target 
than for the untreated cardboard target. 

From this experiment, we concluded that the range means for the black and cardboard targets 
are identical. However, whenever we observe an abrupt change in reflectance across the boundary 
between the black and cardboard targets, we also observe an abrupt change in range (i.e., in the 
left figure, abrupt range changes exist at columns 122 and 135 where the boundaries of two surface 
materials exist). However, in reality, there is no depth discontinuity between the two different 
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Figure A.l: Testing the effect of surface material on range measurements. 




Figure A.2: Mean and standard deviation of range measurements across object boundaries. 
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surfaces. We speculate that the fluctuation in reflectance at the boundary could affect the phase 
difference measurement resulting in a range discrepancy. 

A second change noticed was the effect of temperature on the range measurements. Compare the 
range drift results (Figure 5.10) with Figure 6.9. In the first case, the range measurement is inversely 
proportional to temperature, in latter case, the range is directly proportional to temperature. 
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